Field of the Invention
The present invention relates to a technology for performing a resolution enhancement process and a noise reduction process for an image obtained by image capturing.
Description of the Related Art
In capturing an object image through an image capturing apparatus, such as a digital camera, light emitted from one point on the object cannot converge into one point due to influence of an image capturing optical system (simply referred to as an “optical system” hereinafter), such as a diffraction and an aberration, and has a fine spread. This fine spread is expressed by a point spread function (“PSF”). In obtaining an image (captured image) through image capturing, the PSF is convoluted in the ideal object image. At a result, a blur occurs in the captured image and the resolution of the captured image degraded.
There is proposed a resolution enhancement process that is configured to correct such a degraded captured image using an image processing technology. For example, an image restoration process as one of the resolution enhancement processes is as follows.
Assume that f(x, y) is an original image that has not yet been degraded by an optical system, h(x, y) is a PSF, and g(x, y) is a degraded image or captured image in a real space (x, y). Then, the following expression (1) is established where * is a convolution.g(x,y)=∫∫f(X,Y)*h(x−X,y−Y)dXdY  (1)
When the expression (1) is Fourier-transformed and converted into an expression of a frequency space (u, v), the following expression (2) of a product of frequencies is obtained.G(u,v)=F(u,v)·H(u,v)  (2)
F(u, v) and G (u, v) are those obtained by Fourier-transforming f(x, y) and g(x, y), respectively. H(u, v) is Fourier-transformed h(x, y) or an optical transfer function (“OTF”).
In order to obtain the original image from the degraded image, both sides of the expression (2) may be divided by H(u, v), as illustrated in the expression (3).F(u,v)=G(u,v)/H(u,v)  (3)
The restored image as the original image f(x, y) is obtained by inversely Fourier-transforming F(u, v) or G(u, v)/H(u, v).
In actually obtaining the original image through the image restoring process, a noise contained in the degraded image is remarkably amplified. As the image restoring method for restraining a noise amplification, use of a Wiener filter expressed by the expression (4) is known.
                                          λ            f                    ⁡                      (                          u              ,              v                        )                          =                              1                          H              ⁡                              (                                  u                  ,                  v                                )                                              ⁢                                                                                      H                  ⁡                                      (                                          u                      ,                      v                                        )                                                                              2                                      (                                                                                                              H                      ⁡                                              (                                                  u                          ,                          v                                                )                                                                                                  2                                +                Γ                            )                                                          (        4        )            
In the expression (4), H(u, v) is the OTF, as described above. Γ is a constant for reducing a noise gain, and usable as an adjustment term for controlling the gain. λf(u, v) is a frequency characteristic of a Weiner filter, and represents a frequency characteristic of the resolution enhancement process as the image restoration process.
Since H(u, v) of the actual image capturing optical system is different for each region (position) on the image, the frequency characteristic λf(u, v) of the resolution enhancement process illustrated by the expression (4) may be different for each region on the image. The region on the image contains one pixel.
As illustrated in the expression (5), when λf(u, v) is multiplied by G(u, v), a phase component of the PSF generated by the diffraction, the aberration, etc. of the optical system can be made 0 and the frequency characteristic of the amplitude component can be amplified. This configuration can provide a well restored image with a high resolution. R(u, v) is information of the frequency space of the restored image.R(u,v)=G(u,v)*λf(u,v)  (5)
When the gain is increased by reducing Γ of the Wiener filter, the effect of the resolution enhancement process improves but the noise remarkably amplified. When the gain is reduced by increasing Γ of the Wiener filter, the noise restraining effect improves but the resolution enhancement process becomes less effective.
Japanese Patent No. 5,337,049 discloses a method for performing a noise reducing process for a resolution enhanced image obtained by a resolution enhancement process so as to maintain the effect of the resolution enhancement process and to restrain the noise. This method performs the noise reduction process using a two-dimensional filter, such as a sigma filter and an averaging filter, for the high resolution image. A parameter for the noise reducing process is varied according to the noise level determined by the function of the position on the image.
U.S. Pat. No. 8,427,559 discloses a noise reduction process using a non-local means (“NLM”) filter known as a noise reducing filter that is less likely to degrade an image. This noise reduction process reduces the noise by using self-similarity of an object space and by replacing a pixel value of a target pixel with a weighted mean pixel value of a plurality of partial images similar to the partial image structure contained in the target pixel. A weight used for the weighted mean is determined by a distance between a vector generated with respective pixel values of the partial images around the target pixel as components, and a vector generated with pixel values of pixels around the target pixel as components.
However, the local noise reduction process in the method disclosed in JP 5,337,049 may deteriorate an edge etc. in the image and reduce the effect of the resolution enhancement process. The NLM filter disclosed in U.S. Pat. No. 8,427,559 can reduce the noise while maintaining the resolution, but may degrade the image and cause a residue noise depending on a set parameter.